Adaptive Multi-cue 3D Tracking of Arbitrary Objects

We present a general method for RGB-D data that is able to track arbitrary objects in real-time in challenging real-world scenarios. The method is based on the Condensation algorithm. The observation model consists of a target/background classifier that is boosted from a pool of grayscale, color, and depth features. The training set of the observation model is updated with new examples from tracking and the classifier is re-trained to cope with the new appearances of the target. A mechanism maintains a small set of specialized candidate features in the pool, thus decreasing the computational time, while keeping the performance stable. Depth measurements are integrated into the prediction of the 3D state of the particles. We evaluate our approach with a new benchmark for RGB-D tracking algorithms; the results prove our method to be robust under real-world settings, being able to keep track of the targets over 96% of the time.